Mass CD/DVD Migration: A Novartis Case Study

Nowadays computers are an essential part of modern pharmaceutical research and data storage. It is also well known that experimental and observational data is crucial for scientific research. For approximately twelve years now, the pharmaceutical industry is storing its research data on portable storage media, where each research group was obligated to archive their research data on such portable media but they had also much individual responsibility for the archiving process. So far there are over 3.000 portable media in the raw data archives of Novartis. Digital data archives are more unstable than paper-based archives basically due to development of computer technology and rapid changes of digital storage media. To guarantee the future readability and to preserve the research data, Novartis has decided to migrate the data from the current portable storage media to a storage type more suitable for long-term preservation while preserving the authenticity and provenance. In this paper, we would like to report on this migration project that nicely illustrates several problems of digital archiving and provides state-ofthe-art solutions for them. Introduction Companies in the pharmaceutical industry like Novartis have historically documented their experimental and scientific data in paper-based laboratory notebooks. These notebooks have been afterwards stored in the companies archives for safekeeping. These days computers are an essential part of modern pharmaceutical research and data storage so that today more and more research material is “born digital” [1]. Accordingly, the change from paperbased laboratory notebooks to suitable digital formats has taken place. For about twelve years now, the pharmaceutical industry is storing its research data on portable storage media such as CDs, DVDs, ZIP drives and other kinds of media. Up until now, every research group is obligated to provide all of their research data on portable storage media for the electronic archive but how this had to be done was handled individually. Therefore each research group have been provided with a lot of personal responsibility how to manage the research data. Consequently the raw data archives were and are still growing instantly and rapidly. Up to the present there are over 3000 portable storage media in the raw data archives of Novartis and still growing daily. It is obvious that the era of computers has converted the way of creating, handling, accessing and archiving experimental and scientific data. To guarantee the future readability and to preserve the research data, the data from the current portable storage media needs to be migrated to a storage type more suitable for long-term preservation while preserving the authenticity and provenance. The challenges now lie in the rethinking of the archiving process, while at the same time manage the digital data that has to be preserved. The digital formats in use will inevitably become obsolete with time and need to be migrated. The archiving processes that were created for paper based journals need to be adapted to digital data, and not just taken over. Therefore there is a strong demand for a long-term digital archiving solution. In the following, we would like to report on the Novartis migration project that nicely demonstrates multiple problems of digital archiving and presents a state-of-the-art solution. The remainder of this paper is organized as follows. The next Section discusses the general digital preservation theory and positions of our work inside the preservation process. In Section Migration Description, we describe and discuss the migration process as we developed it for Novartis and the problems we encountered during our work, and the last Section concludes the paper. Digital Preservation Theory We can describe the process of digital preservation as a communication process with the future ([2] and [3]). This process of communication with the future transmits our digital data over four layers as depicted in Figure 1. Beginning with the information that we want to preserve down to the physical medium onto which this information is stored. To be able to preserve information over time, i.e. the digital data that we want to archive, we have three layers that we need to attend to. Storage (t0) Access (t0 + ∆t) Information Preservation Bit Preservation Information Logical Representation